We assessed how synchronous speech listening and lipreading affects speech recognition in acoustic noise. In simple audiovisual perceptual tasks, inverse effectiveness is often observed, which holds that the weaker the unimodal stimuli, or the poorer their signal-to-noise ratio, the stronger the audiovisual benefit. So far, however, inverse effectiveness has not been demonstrated for complex audiovisual speech stimuli. Here we assess whether this multisensory integration effect can also be observed for the recognizability of spoken words. To that end, we presented audiovisual (...) sentences to 18 native-Dutch normal-hearing participants, who had to identify the spoken words from a finite list. Speech-recognition performance was determined for auditory-only, visual-only (lipreading) and auditory-visual conditions. To modulate acoustic task difficulty, we systematically varied the auditory signal-to-noise ratio. In line with a commonly-observed multisensory enhancement on speech recognition, audiovisual words were more easily recognized than auditory-only words (recognition thresholds of -15 dB and -12 dB, respectively). We here show that the difficulty of recognizing a particular word, either acoustically or visually, determines the occurrence of inverse effectiveness in audiovisual word integration. Thus, words that are better heard or recognized through lipreading, benefit less from bimodal presentation. Audiovisual performance at the lowest acoustic signal-to-noise ratios (45%) fell below the visual recognition rates (60%), reflecting an actual deterioration of lipreading in the presence of excessive acoustic noise. This suggests that the brain may adopt a strategy in which attention has to be divided between listening and lipreading. (shrink)

Background Speech understanding may rely not only on auditory, but also on visual information. Non-invasive functional neuroimaging techniques can expose the neural processes underlying the integration of multisensory processes required for speech understanding in humans. Nevertheless, noise (from fMRI) limits the usefulness in auditory experiments, and electromagnetic artefacts caused by electronic implants worn by subjects can severely distort the scans (EEG, fMRI). Therefore, we assessed audio-visual activation of temporal cortex with a silent, optical neuroimaging technique: functional near-infrared spectroscopy (fNIRS). Methods (...) We studied temporal cortical activation as represented by concentration changes of oxy- and deoxy-hemoglobin in four, easy-to-apply fNIRS optical channels of 33 normal-hearing adult subjects and 5 post-lingually deaf cochlear implant (CI) users in response to supra-threshold unisensory auditory and visual, as well as to congruent auditory-visual speech stimuli. Results Activation effects were not visible from single fNIRS channels. However, by discounting physiological noise through reference channel subtraction, auditory, visual and audiovisual speech stimuli evoked concentration changes for all sensory modalities in both cohorts (p<0.001). Auditory stimulation evoked larger concentration changes than visual stimuli (p<0.001). A saturation effect was observed for the audiovisual condition. Conclusions Physiological, systemic noise can be removed from fNIRS signals by reference channel subtraction. The observed multisensory enhancement of an auditory cortical channel can be plausibly described by a simple addition of the auditory and visual signals with saturation. (shrink)

A theory of the joint measurement of quantum mechanical observables is generalized in order to make it applicable to the measurement of the local observables of field theory. Subsequently, the property of local commutativity, which is usually introduced as a postulate, is derived by means of the theory of measurement from a requirement of mutual nondisturbance, which, for local observables performed at a spacelike distance from each other, is interpreted as a requirement of macrocausality. Alternative attempts at establishing a deductive (...) relationship between relativistic causality and local commutativity are reviewed, but found wanting, either because of the assumption of an unwarranted objectivity of the object system (algebraic approach) or because of the use of a projection postulate (operational approach). Finally, the quantum mechanical nonobjectivity is related to certain features of nonlocality which are present in the formalism of quantum mechanics. (shrink)

This work is the product of several years of intense study of the various aspects of Kant's work, and the attempt to provide insights for students both with respect to the details of the Kantian system, and into the development and implications of the system as a whole. During that time many individuals have contributed to its ultimate formulation, and I would like to express my appreciation at least to the more generous contributors. For a careful reading of the manuscript (...) in its earlier forms, and suggestions which helped in many ways to improve the work and to crystalize its thesis, I would like to thank Professors Wilbur Long, A. C. Ewing, and Richard Bosley. For their interest and encouragement in the later stages of the project, I must thank Professor Lewis White Beck, and the many students who have taken my Kant seminar at the University of Alberta, especially Mr. Dieter Hartmetz. And finally, 1 acknowledge with pleasure my longstanding debt to Professor William H. Werkmeister for his years of critical advice and encouragement. Perhaps only Kant and my wife have contributed more to my philosophic development. Acknowledgment must also be made of the permission kindly granted by various publishers for the use of material from the following works under their copyright. Kant's Critique of Practical Reason, translated by Lewis White Beck (copyright 1956, by The Liberal Arts Press, Inc. (shrink)